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ABSTRACT 

Traditionally, students’ feedback in the form of Student Course Evaluation (SCE) has been 
paper-based and made mandatory on students. Even when SCE is made online and 
voluntary, one major obstacle is low response rate. The Non-Constraint Engagement (NCE) 
Model is a newly introduced method in enhancing SCE in our institution, that attempts to 
overcome the common limitations of SCE. This study aims to examine the stability and 
sustainability of the NCE model implementation before, during and after the peak of the first 
wave of the COVID-19 pandemic in the Canadian University Dubai (CUD). The NCE 
Model was initially piloted between 2014 and 2015 and was found to be effective. To test its 
feasibility and sustainability over a longer period of time, an SCE exercise was implemented 
among undergraduate students across four faculties. For analysis, we used SCE data from 
2015 to 2021. Evaluations were performed via Moodle in the online Learning Management 
System (LMS) before mid-term of each semester. There were two domains of SCE: course 
rating and instructor rating. Results showed acceptable and stable response rates, despite 
SCE being voluntary. The COVID-19 pandemic did not cause a fall in student participation. 
Instead, following the outbreak arrival, there was a sharp increase in SCE response rates. 
Similarly, students’ rating on their courses and instructors remains high despite the massive, 
sudden change from physical to online instruction. This study introduces a new approach, the 
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NCE model, which can be tested in other educational settings to enhance SCE. 


1. INTRODUCTION 


Across the globe, higher learning institutions adopt student 
course evaluations (SCE) as a method to assess students’ 
satisfaction level and the effectiveness of teaching and 
learning activities. Feedback from students plays a huge 
role in helping instructors improve course contents, 
delivery method and teaching practices. SCE is also one of 
the factors taken into account by the administration while 
making decisions related to tenure, promotion and pay 
raise [1]. Beyond the conventional uses of SCE, these 
evaluations create a healthier learning environment 
through giving students a sense of belonging, voice and 
agency [2]. Academics on the other hand benefit from SCE 
by deriving motivation to maintain or improve 
performance, and by being held to accountability [3]. 


In recent decades, SCE has largely shifted from the paper- 
and-pen to online mode. This change was triggered by the 
digital revolution and massive uptake of the internet and 
Information and Communication Technology (ICT) that 
have swept across almost all sectors of human life including 
education, business, economy and many more. Various 
studies supported the use of online SCE and found it to be 
more efficient, cost-effective, eco-friendly and flexible [4, 
5]. In addition, online SCE has been demonstrated to be less 
time-consuming, less resource-intensive, and able to gauge 
more response and data in terms of quantity and quality [6]. 
The challenge of online SCE however, lies in getting 
adequate response rates from students [7]. It is a common 
phenomenon worldwide that higher learning institutions 
face difficulties in getting students engaged and interested 
to provide honest feedback. 
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Due to the widespread low response rates, many universities 
make SCE mandatory on students [8,9]. This means students 
are explicitly instructed to fill out SCE, and failure to do so 
comes with some forms of penalty (eg, inability to access 
exam results or academic transcripts, etc). The rationale 
behind mandatory — or authority-based evaluations — is that 
the more responses obtained, the more valid the results. 
However, empirical evidence shows that mandatory SCE 
can easily lead to misinformation bias; approximately 30% 
of students admitted that they gave inattentive responses 
when SCE was made compulsory [10]. Similarly, studies 
also found that imposing SCE on students did not increase 
the reliability or meaningfulness of feedback given [10]. On 
the other hand, voluntary SCE is increasingly seen as being 
more progressive, flexible and student friendly. 
Nevertheless, the drawbacks include poor response and lack 
of sample representativeness, which in turn may affect the 
validity of results. To mitigate this, universities invest huge 
resources in developing methods and strategies to make SCE 
online and voluntary, while giving high response rates. 


With the advent of the COVID-19 pandemic, universities 
worldwide have faced massive disruptions to their academic 
schedules and undergone a major change in the mode of 
learning [11]. With closures of learning institutions as a 
result of physical distancing measures, communication 
between lecturers and students, or between administrative 
staff and students, have become largely virtual. Prior to the 
pandemic, adequate response rates in voluntary SCE are 
generally driven by visible and face-to-face initiatives such 
as verbal reminders in classes, and posters or notice boards 
in campus. With these physical elements now not in place, 
the university administration has to depend solely on virtual 
reminders and pleas (eg, through emails or university 
websites). The effectiveness of these online reminders in 
engaging students is difficult to determine. 


This study aims to test the stability and sustainability of the 
Non-Constraint Engagement (NCE) model in SCE before, 
during and after the peak of the first wave of the COVID-19 
pandemic, at the Canadian University of Dubai in United 
Arab Emirates [12]. More specifically, our objectives are: 


1. To determine if the NCE model can maintain 
similar response rates during the first wave of the COVID- 
19 pandemic. 

2, To study the impact of COVID-19 on the trend of 
evaluation scores related to course and instructor rating. 
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23 METHODOLOGY 


A. The Non-Constraint Engagement (NCE) Model 


The Canadian University of Dubai (CUD) adopted the Non 
Constraint Engagement (NCE) Model in its evaluation 
system, which was initiated and piloted from 2014 to 2015. 
The NCE is a specific SCE model that is defined as 
“participation of key stakeholders in a structured non- 
constraint framework of student course evaluation 
implementation” [12]. In other words, this model uses an 
online and voluntary approach, while including students as 
key stakeholders in order to drive their engagement. There 
are three major components of NCE: 1) engagement; 2) 
structured non-constraint implementation framework, and; 
3) Result to Action (R2A) [12]. The diagram below 
illustrates the NCE model components and its basic features: 


Three major components. 


Enhancing Student Course Evaluation: 
Non Constraint Engagement (NCE) Model 


3. Results to Action 


1. Engagement 


Figure 1: Non-Constraint Engagement Model (NCE) 


Engagement refers to the active and dynamic participation 
of three key players — students, school and supporting units 
— towards a shared goal. Structured Non-Constraint 
Implementation on the other hand refers to the framework 
and process by which SCE occurs. These entail several 
steps including development and regular improvement of 
the tool (questionnaire), communication of the objectives 
and importance of survey to all stakeholders, undertaking 
of evaluation by students in a voluntary manner, and 
conveying SCE findings to the higher authority. Result to 
Action (R2A) is an outcome of the two previous 
components in which identified loopholes are rectified or 
addressed [12]. 


B. Study setting, study period and data collection 


After a successful pilot project in 2014/2015, the NCE 
model went into a full implementation phase from 2016 to 
2019. However, its execution and effectiveness were 
doubted when the COVID-19 outbreak towards the end of 
2019 (followed by global movement restrictions in early 
2020). This is because the NCE model depends heavily on 
constant motivation and moral support offered to students 
through interactions in the campus that include on-ground 
campaigns by the Student Affairs Division, verbal 
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reminders in classes, posters and notice boards. The 
closure of CUD campus, along with the shift of 
communication mode to online platforms (reminders were 
then mainly sent via emails) were expected to have a 
negative impact on students’ motivation to undertake SCE. 
This could be further compounded by other effects of 
COVID-19 on students’ psychological status as a result of 
sudden disruptions to classes and exam schedules, travel 
restrictions, economic downturns and the widespread fear 
and uncertainty. 


To test the stability and sustainability of the NCE model, 
we used data from SCE across all undergraduate and 
postgraduate faculties. These include the Faculty of 
Applied Science and Technology (FEAST), Faculty of 
Architecture and Interior Design (FAID), Faculty of 
Management (FOM) and Faculty of Communication, Arts 
and Sciences (FCAS). SCE was performed via Moodle in 
the online Learning Management System (LMS), starting 
from 2015/16 until 2020/21 in every Fall and Spring 
semester. The SCE survey forms has two domains: course 
rating and instructor rating. Each consists of 10 items, with 
response options provided in a scale of 1 to 5; 1 indicates 
the lowest score and 5 indicates the highest. 


3. RESULTS 


Response rates were found to be fairly stable throughout 
the period of five years between 2015 and 2019, despite 
the minimal year-to-year fluctuations. However, there was 
a sharp increase of response rates across all the four 
faculties in the last year, between 2019/2020 and 
2020/2021. This corresponded with the advent (and 
continuation) of COVID-19 outbreak. Overall, the average 
response rates for FCAS, FOM, FAID and FEAST were 
41%, 40%, 45% and 45% respectively, with the 
percentage being lower in Spring semesters. 


The figure below illustrates the trend of response rates in 
the last six years, according to faculty. 


Response rates (%) in SCE from 2015 to 2021 using the NCE model 


Fall Spring | Fall Spring | Fall Spring | Fall Spring | Fall Spring | Fall Spring 
2015/16 2016/17 2017/18 2018/19 2019/20 2020/21 
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Figure 2: Trend of response rates in SCE between 2015 and 2021 
using the NCE model 


In the beginning of 2015/2016, response rates ranged 
between 44% and 60% among the four faculties. Toward 
the end of 2019/2020, they were between 30% to 39%, 
reflecting a downward trend. Some fluctuations could be 
seen in the interval period between the former and latter. 
By the end of 2019 and early 2020 — corresponding with 
the advent of the COVID-19 pandemic — a significant leap 
was visible as response rates jumped to 61% to 80% in Fall 
semester, followed by 68% to 74% in Spring semester. 
Analysis according to each faculty shows that FOM has a 
relatively low trend of response rate compared to the other 
three faculties, while FAID demonstrates the highest 
constant trend of response rates. 


The figure below presents the overall (average) trend of 
response rates in SCE in the last six years, with all faculties 
combined. 


Overall 


Fall Spring | Fall Spring | Fall Spring | Fall Spring | Fall Spring Fall Spring 
2015/16 2016/17 2017/18 2018/19 2019/20 2020/21 


Figure 3: Trend of response rates in SCE between 2015 and 2021 
using the NCE model 


On the average, response rates ranged between 31% to 
52%, prior to the pandemic. Fluctuations point to a 
common pattern of higher response rates in Fall semester 
followed by a drop in Spring semester, with the exception 
of the year 2016/2017. In early 2020 (as COVID-19 
struck), there was a huge increase in response rates, by 
almost 40%. Between Fall and Spring semesters in that 
same (last) year, response rates plateaued. 


A. Trend of students’ rating 


With regards to course and instructor rating, evaluation 
scores were largely above the minimum targeted value, 
which was 4.0, with the exception of two data points for 
course rating, which were 3.97 and 3.94 in 2015/16 Fall 
and Spring semester, respectively. Following that year, 
scores were consistent higher than 4 across all the 
faculties. The period immediately following COVID-19 
arrival did not show any decline in evaluation scores. 
Instead, the scatter plot demonstrates a gradual upward 
trend of evaluation scores from 2015 to 2021. 
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The two figures below show the distribution of evaluation 
scores obtained from SCE across two domains; course 
rating and instructor rating. 


Evaluation scores on course rating using the NCE model 
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Figure 4: Scatter plot showing evaluation scores of course rating 
between 2015 and 2021 


Evaluation scores on instructor rating using the NCE model 
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Figure 5: Scatter plot showing evaluation scores of instructor 
rating between 2015 and 2021 


*in the x-axis, each interval represents a period of one year 


*blue arrow indicates the period around the advent of COVID- 
19 pandemic 


When scores of both domains (course and instructor 
rating) were combined, mean values showed an overall 
slight upward trend from 2015 to 2021. The pandemic 
advent did not cause any drop in the average evaluation 
score, but maintained (or slightly increased) it. The figure 
below illustrates the trend of mean (average) evaluation 
score for both domains (content and instructor rating) in 
the period of six years. 


Overall SCE scores (rating) from 4 faculties 
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Figure 6: Trend of mean evaluation scores for both SCE domains 
from 2015 to 2021. 


4. DISCUSSIONS 


The overarching aim of this paper was to test the stability 
and sustainability of the NCE model in SCE in terms of 
response rates and students’ rating. More specifically, we 
wanted to determine whether (or not) the NCE model 
could maintain high response rates over the years, despite 
the impact of COVID-19. Likewise, we also attempted to 
study the trend of evaluation scores over the same period, 
while checking if students’ rating changed as a result of 
the pandemic and its related counter measures. Our initial 
hypotheses were that response rates and evaluation scores 
would drop significantly following the arrival of COVID- 
19 in UAE - especially from March 2020 onward when its 
containment measures began, including lockdowns and 
closures of learning institutions. 


Overall, our findings showed that prior to COVID-19, 
response rates were maintained between 30% and 50%, 
with fluctuations in between. There was a pattern of 
cyclical drop in the Spring semester every year, except for 
2016/2017. This range was in line with findings from 
previous empirical studies that measured response rates in 
online student evaluation surveys. For instance, Chapman 
et al. reported an online response rate of 43% [13] while 
Avery et al (2006) and Nulty (2008) found the response 
rates between 30 % to 40% [14, 15]. Our results however 
were higher than those of by Ling et al (2012) who 
reported online response rates of less than 30% [16]. 


The COVID-19 pandemic did not seem to negatively 
affect students’ participation in SCE in the four faculties. 
Instead, our results demonstrated a sharp increase in 
response rates, indicating that more students showed 
interest to provide feedback despite the challenges they 
faced in terms of mobility and disruption to academic 
schedules. Schools and universities in the UAE were 
closed on 8 March 2020, prompting learning institutions 
across the country — including CUD — to undergo a major 
shift in its mode of teaching delivery, by adopting 
Emergency Remote Teaching (ERT) [17, 18]. Such a 
sudden change must have had a huge impact on students’ 
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ability to cope academically and emotionally, given that 
ERT was new and temporary measure, compared to 
traditional face-to-face sessions [19, 20]. This impact 
could be further aggravated by the relative inexperience 
among lecturers in navigating the technicalities of virtual 
platforms and ensuring students’ meaningful participation 
from behind screens [21]. Despite the merits of online 
learning, prior studies have highlighted a number of 
possible drawbacks such as low level of engagement by 
students due to limited non-verbal cues, the higher need 
for self-discipline and monitoring, and fewer chances to 
make meaningful interactions with peers [22, 23]. 


The leap in response rates following COVID-19 arrival — 
which also corresponded with the period of massive 
uptake of virtual classes — can be explained by several 
reasons. First, the NCE model which emphasizes student 
engagement as key stakeholders could have successfully 
sustained their interest to participate regardless of the 
change in learning environment. This is due to the model’s 
‘student-centered’ approach that is specifically designed to 
impart a sense of ownership and belonging, which in turn 
make them view SCE as a responsibility and platform for 
change. Second, even though reminders were sent mainly 
through emails following the closure of CUD campus, the 
administration made sure that clear and powerful 
messages were communicated and students were made 
aware of the value and impact of their feedback. Third, the 
rapid vaccination roll-out in UAE which began in early 
January 2021 may have brought a sense of hope for 
‘normalcy’ to people, as restrictions were gradually eased 
[24, 25]. This could have increased students’ motivation 
and optimism and affected their level of engagement with 
SCE. However, the association between COVID-19 
vaccination and high response rates remains hypothetical, 
as we did not have additional, relevant data. Even if this 
was true, the impact of vaccination could be only linked to 
the maintenance of high response rates (between Fall and 
Spring 2020/2021) and not to the sharp increase prior to 
that. 


From a different perspective, the significant rise in 
response rates can be due to a possible ‘pseudo-effect’ of 
the pandemic, characterized by an initial ‘over-caution’ by 
the university administration. As a result of anxiety over 
the possibility of fall in SCE response rates, the unit in 
charge may have doubled their effort in reaching out to 
students and reminding them about SCE, which in turn 
drove up the number of respondents. Similarly, on the 
students’ side, given their first experience with a massive 
disease outbreak, this might have led to a sense of ‘hyper- 
awareness’ in the first few weeks or months following 
COVID-19 arrival. Such hyper-awareness can manifest in 
the form of ‘hyper-connectedness’ to the internet and 
therefore more responsiveness to SCE. Likewise, students 


may show extra enthusiasm with the newly experienced 
ERT and thus feel eager to provide feedback. However, 
this ‘pseudo-effect? may not last long, especially when 
fatigue toward COVID-19 counter-measures and online 
learning sets in and student become desensitized. When 
that happens, response rates can gradually fall to the earlier 
baseline. 


As regards course and instructor rating, evaluation scores 
showed a slight upward trend between 2015 and 2018, 
followed by a plateau between 2019 and 2021. The 
pandemic did not cause any drop in the students’ rating on 
their courses and instructors. This was in line with the 
findings by Boysen (2020) [26]. Sustenance of modest 
evaluation scores could imply that lecturers and the 
university administration managed the transition period 
well and have successfully adjusted to ERT, while 
maintaining a somewhat similar degree of effectiveness. 
However, it is also crucial to look into the SCE 
questionnaire items, as the content of questions influences 
respondents’ answers. For instance, if the items do not 
mention anything related to the shift from physical to 
online classes and how this change affects students, 
respondents may perceive it as irrelevant while providing 
their feedback. In the CUD SCE survey, there are two 
items in the course rating that can capture information 
relevant to how COVID-19 may have affected students: a) 
item 5 inquires students’ level of motivation, and; b) item 
8 inquires on the learning environment. The remaining 
items are rather general and unlikely to change pre- and 
post-COVID-19. This shows the importance of analysing 
selected items within the SCE questionnaire, as focusing 
only on overall evaluation scores can easily mask ‘small 
findings’ which are more meaningful for improvement. 


This study has several limitations. First, despite the modest 
response rates throughout the study timeframe (from 2015 
to 2021), we did not test whether samples have been truly 
representative of the student population, which can raise 
the issue of non-response bias [27]. Prior studies have 
consistently showed that students who are more likely to 
engage in SCE are those with higher academic grades and 
level of motivation. 


On the other hand, students whose academic performance 
and motivation level are poor tend to show less interest in 
giving feedback [10, 28]. However, based on the minimum 
response rates required as suggested by Nulty (2008) [15], 
our response rates were more than adequate to produce 
reliable results due to the big enrolment size (data not 
shown). 


Second, our analyses were restricted to quantitative data. 
We did not include the qualitative responses obtained 
(there is one open-ended question at the end of the survey 
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which has not been taken into account in this paper). This 
may have given an incomplete picture to our findings and 
missed some valuable input that can complement our 
quantitative findings [29]. 


Third, the SCE data after the advent of COVID-19 is 
limited to one year, or two rounds of evaluations. 
Therefore, we could not ascertain if the NCE model could 
sustain the leap in response rates after the sharp increase 
in 2019/2020, and whether this leap was a ‘pseudo-effect’ 
that would fade with time. 


5. CONCLUSION 


From the study findings, we can conclude that the NCE 
Model is stable and sustainable. The model has 
successfully maintained adequate response rates four to 
five years prior to COVID-19 pandemic and continued to 
appear robust during the crisis peak. This was evidenced 
by the huge leap in response rates in the period 
immediately after COVID-19 outbreak began — contrary 
to our original hypotheses — despite all the unexpected 
changes that took place within the learning environment. 
Many factors could have contributed to this phenomenon, 
and we were unable to capture all possible variables or 
confounders. However, available data demonstrates that 
the current SCE system using the NCE model is 
functioning well and has achieved one of its main 
objectives, that is to engage students as key stakeholders 
and impart in them a sense of ownership in this process. 
We also found that evaluation scores remained high and 
stable, despite the pandemic effects. Overall, our findings 
should be interpreted in the light of the above-mentioned 
limitations. We recommend that CUD continues to 
monitor and analyze SCE data in the next several years to 
ascertain if the significant increase in response rates post- 
COVID-19 can be maintained. 
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